In AmandaPMurphy/SemesterProjectLabII:

knitr::opts_chunk$set(echo = TRUE)

1. Section 12.3.3 from your textbook refers to: The problem with replications of a meaningless experiment: ‘alpha and the captain’s age’. The issue here is that if you run an ineffectual experiment enough times you can always find a significant result by chance. The textbook mentions that if you repeat an experiment 20 times, you are guaranteed to find a significant result with .64 probability, and the probability is .92 if you repeat the experiment 50 times. Make use of the rbinom() function to show you can reproduce both probabilities. (1 point)

A <- replicate(10000,sum(rbinom(20,1,.05)))
hist(A)

length(A[A > 0])/10000

B <- replicate(10000,sum(rbinom(50,1,.05)))
hist(B)

length(B[B > 0])/10000

B. If the ineffectual experiment was conducted 20 times, and there were four groups, and the experimenter would accept a significant result from any of the orthogonal linear contrasts, what would be the probability of finding a significant result here? (1 point)

D <- replicate(10000,sum(rbinom(20,3,.05)))
length(D[D > 0])/10000

hist(D)

1. Consider that a researcher publishes a study showing a significant effect, p <. 05; but, in reality the researcher makes a type I error, and the manipulation did not cause any difference. If many other researchers replicated the study, what kind of p-values would they find? Use R to create a sampling distribution of p-values that would be expected in this situation. What shape does this distribution have? (2 points)

library(tibble)

my_pvalues <- c()

for(i in 1:100){


IV <- rep(1:2,each=20)
DV <- c(rnorm(20,0,1), rnorm(20,0,1))

sim_data <- tibble(IV,DV)

my_pvalues[i] <- t.test(DV~IV, var.equal = TRUE, data= sim_data)$p.value

t.test(DV~IV, var.equal= TRUE, data = sim_data)
}

hist(my_pvalues)

1. Now assume that the published result reflects a true effect. Specifically, let’s imagine the study had two groups (between-subjects), with 20 subjects in each group. Assume that scores for subjects are all sampled from a normal distribution, and that group A has larger mean than group B by .5 standard deviations (e.g., Cohen’s d = .5). If many other researchers replicated the study, what kind of p-values would they find? Use R to create a sampling distribution of p-values that would be expected in this situation. What shape does this distribution have? (2 points)

my_pvalues <- replicate(10000, t.test(rnorm(20,.5,1),rnorm(20,0,1),
                                      var.equal=TRUE)$p.value)
hist(my_pvalues)

AmandaPMurphy/SemesterProjectLabII documentation built on May 23, 2022, 3:16 p.m.